OpenAIÂ’s new image analysis update for its chatbot is both impressive and frightening. HereÂ’s how to use it, and some advice for your experiments.
OPENAI RECENTLY ANNOUNCED an upgrade to ChatGPT (Apple, Android) that adds two features: AI voice options to hear the chatbot responding to your prompts, and image analysis capabilities. The image function is similar to what’s already available for free with Google’s Bard chatbot.
Even after hours of testing the limits and capabilities of ChatGPT, OpenAI’s chatbot still manages to surprise and scare me at the same time. Yes, I was quite impressed with the web browsing betaoffered through ChatGPT Plus, but I remained anxious about the tool’s ramifications for people who write for money online, among many other concerns. The new image feature arriving for OpenAI’s subscribers left me with similarly mixed feelings.
While I’ve not yet had the opportunity to experiment with the new audio capabilities (other great reporters on staff have), I was able to test the soon-to-arrive image features. Here’s how to use the new image search coming to ChatGPT and some tips to help you start out.
While the update is expected to arrive before the end of the year, it’s unclear exactly when the image and voice features will roll out to the public. As with most of OpenAI’s updates, like the GPT-4 version of ChatGPT, paying subscribers are first in line to receive access.
There are three ways to upload photos in the ChatGPT mobile app. First, you can select the camera option located to the left of the message bar and take a fresh photo with your smartphone. Before uploading the image, you can use your finger to draw a circle around whatever you’d like the chatbot to focus on.
You can also choose photos from your device to upload and select files that are saved to your phone. Desktop browser users of ChatGPT can just upload saved photos from their computer. Even though there’s no option yet to upload videos to the chatbot, you can submit multiple images in a single prompt.
.jpg)
This isn’t the first time a kind of “computer vision” has been made available to the public, but the accessible user interface paired with a powerful chatbot leaves the impression that there might be something distinct, and potentially transformative, happening here. Before we go any further, remember to never upload personal, sensitive photos to ChatGPT as you try out the image feature.
Want to limit how long OpenAI stores your data and AI interactions to train its chatbot? Go into Settings, then Data Controls, and turn off Chat History & Training. With this disabled, your information is deleted after one month. This process must be done individually for every browser that you use to access ChatGPT, for both PC and mobile.
I received the best results from ChatGPT when uploading images that were clear and well lit. It fumbled a few guesses, but the chatbot was able to label many of the objects in my apartment: from an orchid plant and international coins to a stray charging cable and Steve Irwin Funko Pop.
Despite the tool's power for searching through information, refrain from immediately trusting its answers. ChatGPT mislabeled my daily multivitamin as a pill for treating erectile dysfunction.
.jpg)
One of the major guardrails OpenAI put around this new feature is a limit on the chatbot’s ability to answer questions that identify humans. “I’m programmed to prioritize user privacy and safety. Identifying real people based on images, even if they are famous, is restricted in order to maintain these priorities,” ChatGPT told me. While it didn’t refuse to answer every question when shown pornography, the chatbot did hesitate to make any specific descriptions of the adult performers, beyond explaining their tattoos.
It’s worth noting that one conversation I had with the early version of ChatGPT’s image feature seemed to skirt around part of the guardrails put in place by OpenAI. At first, the chatbot refused to identify a meme of Bill Hader. Then ChatGPT guessed that an image of Brendan Fraser in George of the Jungle was actually a photo of Brian Krause in Charmed. When asked if it was certain, the chatbot switched over to the correct response.
In this same conversation, ChatGPT went wild trying to describe an image from RuPaul’s Drag Race. I shared a screenshot of Kylie Sonique Love, one of the drag queen contestants, and ChatGPT guessed that it was Brooke Lynn Hytes, a different contestant. I questioned the chatbot’s answer, and it proceeded to guess Laganja Estranja, then India Ferrah, then Blair St. Clair, then Alexis Mateo.
“I apologize for the oversight and incorrect identifications,” ChatGPT replied when I pointed out the repetitiveness of its wrong answers. As I continued the conversation and uploaded a photo of Jared Kushner, ChatGPT declined to identify him.
If the guardrails are removed, either through some kind of jailbroken ChatGPT or an open source model released in the future, the privacy implications could be quite unsettling. What if every picture taken of you and posted online was easily tied to your identity with just a few clicks? What if someone could snap a photo of you in public without consent and instantly find your LinkedIn profile? Without proper privacy protections remaining in place for these new image features, women and other minorities are likely to receive an influx of abuse from people using chatbots for stalking and harassment.
https://www.wired.com/story/chatgpt-plus-image-feature-openai/